ON CONTINUOUS TIME MARKOV GAMES WITH COUNTABLE STATE SPACE

On Continuous Time Markov Games with Countable State Space

This paper is a continuation of our papers [61 and [71 and is concerned with a continuous time Markov game in which the state space is countable and the action spaces of player I and player n are compact metric spaces. In the game, the players continuously observe the state of the system and then choose actions. As a result, the reward is paid to player I from player n and the system moves to a...

On Markov Chains with Continuous State Space

In this expository paper, we p r o ve the following theorem, which m a y be of some use in studying Markov chain Monte Carlo methods like hit and run, the Metropolis algorithm, or the Gibbs sampler. Suppose a discrete-time Markov c hain is aperiodic, irreducible, and there is a stationary probability distribution. Then from almost all starting points the distribution of the chain at time n conv...

Uniform CLT for Markov chains with a countable state space

Let (S,G, P ) be a probability space and let F be a set of measurable functions on S with an envelope function F finite everywhere. Let X1, X2, ... be a strictly stationary sequence of random variables with distribution P , and define the empirical measures Pn, based on {Xi}, as Pn = n−1 ∑n i=1 δXi . We say the uniform CLT holds over F , if n 1 2 (Pn − P ) converges in law, in the space l∞(F ) ...

Property-Driven State-Space Coarsening for Continuous Time Markov Chains

Dynamical systems with large state-spaces are often expensive to thoroughly explore experimentally. Coarse-graining methods aim to define simpler systems which are more amenable to analysis and exploration; most current methods, however, focus on a priori state aggregation based on similarities in transition rates, which is not necessarily reflected in similar behaviours at the level of traject...

Countable-State, Continuous-Time Dynamic Programming with Structure